Skip to content

Add opt-in BM25 tool search for large catalogs - #1449

Open
jokemanfire with Copilot wants to merge 4 commits into
mainfrom
copilot/feat-built-in-tool-search
Open

jokemanfire with Copilot wants to merge 4 commits into
mainfrom
copilot/feat-built-in-tool-search

Conversation

Copilot AI commented Nov 12, 2026 •

Copy link
Copy Markdown

Large tool catalogs consume model context and reduce tool-selection accuracy. This adds server-side discovery and a standalone search API for gateways, behind a dependency-free tool-search feature.

  • Search index: Lazy BM25 Okapi ranking over names, titles, descriptions, and optional parameter metadata. Returns full tool definitions and scores; supports catalog replacement.
  • Server wrapper: ToolSearchServer replaces discovery with search_tools / call_tool, supporting always-visible tools, configurable names, and a default five-result limit.
  • Behavior: Original tools remain directly callable. Dispatch rejects synthetic-tool recursion and preserves metadata and retry state. Each listing/search fetches all catalog pages; the index rebuilds only when definitions change.
  • Scope: Delegates non-tool handler behavior unchanged. Includes API documentation; optional regex search is deferred.
use rmcp::tool_search::{ToolSearchIndex, ToolSearchServer};

let index = ToolSearchIndex::new(tools);
let hits = index.search("create a github issue", 5);

// Alternatively, expose synthetic discovery tools:
let server = ToolSearchServer::new(my_handler);

Search controls discovery, not authorization; underlying handlers retain access enforcement.

Copilot AI changed the title [WIP] Add built-in tool search (BM25) for FastMCP Add opt-in BM25 tool search for large catalogs Nov 12, 2026
Copilot AI requested a review from jokemanfire November 12, 2026 14:41
@github-actions github-actions Bot added T-documentation Documentation improvements T-test Testing related changes T-config Configuration file changes T-core Core library changes T-handler Handler implementation changes labels Nov 14, 2026
@jokemanfire
jokemanfire marked this pull request as ready for review November 14, 2026 06:11
@jokemanfire
jokemanfire requested a review from a team as a code owner November 14, 2026 06:11
Copilot AI balanced review requested due to automatic review settings November 14, 2026 06:12
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-17T06:08:23.469446Z be968f5 New commits
🔒 Security Review ✅ Completed 2026-11-17T06:11:05.487327Z be968f5 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 28b11e4a71

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +131 to +132
result.tools.extend(page.tools);
result.next_cursor = page.next_cursor;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve restrictive cache metadata across pages

When a paginated catalog uses different SEP-2549 caching metadata per page, this aggregation retains only the first page's ttl_ms and cache_scope while discarding those fields from every later page. For example, if page one is cacheable/public but a later page containing an always_visible user-specific tool is private or immediately stale, the combined tools/list response remains public/cacheable and may be reused across users. Clear the aggregate's caching fields or merge all pages using the minimum TTL and most restrictive scope.

Useful? React with 👍 / 👎.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Merged tool listings ignore later pages’ cache restrictions, potentially exposing private tool definitions to other users through publicly cacheable responses.

3 open issues
What changed in this PR

Adds opt-in BM25 tool discovery to rmcp, helping servers and gateways expose large catalogs without sending every tool definition to the model.

Changes:

  • Introduces a lazy search index with ranked results, optional parameter indexing, and catalog replacement.
  • Adds a configurable server wrapper exposing search and dispatch tools while preserving direct calls and non-tool behavior.
  • Adds feature-gated tests and API usage documentation without new dependencies.
File Description
crates/​rmcp/​tests/​test_tool_search.rs Tests ranking, discovery, dispatch, refresh, and router integration.
crates/​rmcp/​src/​tool_search/​server.rs Implements configurable discovery and dispatch wrapping.
crates/​rmcp/​src/​tool_search.rs Implements lazy BM25 indexing and ranked search.
crates/​rmcp/​src/​lib.rs Exports the feature-gated search module.
crates/​rmcp/​src/​handler/​server.rs Extends handler delegation for the search wrapper.
crates/​rmcp/​README.md Documents search APIs, configuration, and behavior.
crates/​rmcp/​Cargo.toml Declares the feature and enables it for documentation.

Review effort: Balanced

Comment on lines +131 to +132
result.tools.extend(page.tools);
result.next_cursor = page.next_cursor;

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged cache hints conservatively across catalog pages, taking the minimum available TTL and preferring private scope. Added a regression assertion. Fixed in b6817fc.

Comment thread crates/rmcp/src/tool_search.rs Outdated
let mut frequencies = HashMap::new();
for tool in &self.tools {
let mut text = tool.name.to_string();
for field in [tool.title.as_deref(), tool.description.as_deref()]

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indexing now falls back to ToolAnnotations.title when Tool.title is absent, with a regression test. Fixed in b6817fc.

Comment thread crates/rmcp/src/tool_search/server.rs Outdated
Comment on lines +220 to +227
let args: Search = serde_json::from_value(json!(request.arguments))
.map_err(|e| ErrorData::invalid_params(e.to_string(), None))?;
let limit = args.max_results.unwrap_or(self.config.max_results);
if limit == 0 || limit > self.config.max_results {
return Err(ErrorData::invalid_params(
"max_results is outside the configured range",
None,
));

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Invalid search/dispatch arguments and synthetic dispatch targets now return CallToolResult::error; the recursion guard remains, and tests assert is_error. Fixed in b6817fc.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b6817fc334

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +131 to +134
result.ttl_ms = match (result.ttl_ms, page.ttl_ms) {
(Some(current), Some(next)) => Some(current.min(next)),
(current, next) => current.or(next),
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Account for time spent fetching later pages

When later catalog pages are slow to produce, each page's TTL begins aging before the combined response is returned, but taking the raw minimum resets that lifetime at the end of aggregation. For example, if the first page has a 5-second TTL and subsequent pages take 10 seconds, the wrapper still advertises another 5 seconds even though the first page is already stale. Track when each page was obtained and subtract its elapsed age when computing the aggregate TTL.

Useful? React with 👍 / 👎.

@jokemanfire jokemanfire left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at b6817fc from the code, not the description.

The three earlier findings do hold as fixed

  • Cache hints (crates/rmcp/src/tool_search/server.rs:131-143): minimum TTL across pages, private sticky. The mock returns 10s/public for page 0 and 0/private afterwards (crates/rmcp/tests/test_tool_search.rs:136-141), and :200-201 asserts Some(0) + Private. Matches what was claimed.
  • Tool error instead of JSON-RPC error (server.rs:233-236, :261-267, helper :275-277): deserialization failures, out-of-range max_results and synthetic dispatch targets all return CallToolResult::error; the recursion guard is still there. tests.rs:238, :253, :262 assert is_error.
  • ToolAnnotations.title fallback (crates/rmcp/src/tool_search.rs:115-119): yes, .or_else on the annotation title when Tool.title is absent.

Two things I would fix before merge

1. catalog() has no page cap — a server that never repeats a cursor still loops forever

server.rs:112-146 walks next_cursor until it is None and only rejects repeated cursors (the cursors set at :113-120). Distinct cursors are unbounded by construction: the spec forbids interpreting the value, and there is no counter at :114. One line in a mock — tools: take(1), next_cursor: Some((offset + 1).to_string()) — is an infinite loop that keeps growing result.tools, and this path runs for every downstream that forwards a third-party catalog on every list/search.

This is the same failure mode as #1456 on the client helpers, which is why I would rather not add a second unbounded walk to the same crate. Please bound it (max_pages on ToolSearchConfig, or reuse the client cap constant) and add the endless-cursor mock as a test: today the only termination covered is next_cursor: None (tests.rs:133), so the repeated-cursor guard itself is never exercised either.

2. A page that omits cacheScope/ttlMs is promoted to public/fresh

:135-143 maps (None, Some(Public)) to Public, and :131-134 maps (None, Some(ttl)) to that TTL. Those fields are optional because peers older than 2026-07-28 never send them — see the ttl_ms doc on ReadResourceResult in model.rs ("Required by spec version 2026-07-28, but optional here to maintain compatibility with older spec versions"). An absent hint means that page promised nothing, not that it promised another page's freshness. Merging mixed pages to Public lets a shared cache serve the unhinted page's tools (including always_visible ones) to other users. Conservative merge: None, or Private, if any page left the field absent.

3. TTL aging — codex's 4578114101 holds

:131-134 takes the raw minimum of per-page TTLs, but each TTL starts when that page was received. After a 10s second page, a 5s page-0 TTL is advertised as 5s measured from the aggregate's timestamp. Either record the per-page receive time and subtract elapsed age, or document that the merged TTL is an upper bound.

I have not approved: 1 and 2 change behaviour and need a test each. Style and the wrapper-macro refactor in handler/server.rs look fine to me — the Box/Arc impls keep the exact previous bodies.

Copilot AI requested a review from jokemanfire November 17, 2026 02:52
@jokemanfire

Copy link
Copy Markdown
Member

@copilot fix the ci lint

Copilot AI commented Nov 17, 2026

Copy link
Copy Markdown
Author

@copilot fix the ci lint

I checked the CI run: Clippy passed; the failing check is Lint Commit Messages. It rejects all three PR commits because their subjects lack Conventional Commit types. I corrected the subjects locally, but the branch update attempt failed during rebase/signing and the PR head remains unchanged. This needs the commit-message rewrite to be applied to the remote branch.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be968f513c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

.await?;
result.ttl_ms = match (result.ttl_ms, page.ttl_ms) {
(Some(current), Some(next)) => Some(current.min(next)),
(current, next) => current.or(next),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat an omitted page TTL as immediately stale

When one page omits ttlMs and another advertises a positive TTL, current.or(next) retains the positive value, so the combined response allows caching tools from the page that supplied no lifetime. For a 2026-07-28 peer, the ordinary handler path would normalize that omitted hint to 0, but these inner list_tools calls bypass that normalization; preserve the omission or merge it as zero so the outer response becomes immediately stale.

Useful? React with 👍 / 👎.

Comment on lines +139 to +140
} else if result.cache_scope.is_some() || page.cache_scope.is_some() {
Some(CacheScope::Public)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep an omitted page scope from becoming public

When one page declares Public and another omits cacheScope, this branch promotes the aggregate to Public. The normal 2026-07-28 server response path uses ServerResult::fill_missing_cache_hints to make an omitted scope Private, but the wrapper calls the inner handler directly, so the omitted page never receives that protection and its tools can be cached across users. Leave the merged scope absent when either non-private page omits it, allowing final response normalization to choose Private.

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T-config Configuration file changes T-core Core library changes T-documentation Documentation improvements T-handler Handler implementation changes T-test Testing related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: built-in tool search (BM25) for large tool catalogs, like FastMCP's search_tools

3 participants